Hippocampal Spatial Model for State Space Representation in Robotic Reinforcement Learning

We study reinforcement learning for cognitive navigation. The state space representation is constructed by unsupervised learning during exploration. As a result of learning, a stable representation of the continuous two-dimensional manifold in the high-dimensional input space is found. The representation consists of a population of localized overlapping place fields. This statespace coding is a biologically inspired model of place fields in the Hippocampus of the rat. Place fields are learned by extracting spatiotemporal properties of the environment from visual sensory inputs. Visual ambiguities are eliminated by taking into account self-motion signals via path integration. This solves the hiddenstate problem and provides a robust representation suitable for applying Q-learning in continuous space. Reward-based function approximation takes place to drive action units one synapse downstream from place cells. The teaching error models the dopaminergic reward-expectation signal from neurons in the brainstem. Several action modules share the same space representation and guide the robot to multiple targets. The experimental validation of the system is done by implementing it on a mobile Khepera robot.